Efficient Peer-To-Peer Searches Using Result-Caching

Existing peer-to-peer systems implement a single function well: data lookup. There is now a wealth of research describing how to reliably disseminate, and to later retrieve, data in a scalable and load-balanced manner. However, searching has received less attention. The current state of the art is to distribute inverted indexes in the name space. Intersection of distributed sets can be made more efficient by exchanging bloom filters prior to moving objects [2]. This paper proposes an orthogonal and complementary technique: using result-caching to avoid duplicating work and data movement. For example, assume that indexes ai, aj , and ak are located on distinct nodes in the network. Computing ai ^ aj ^ ak directly from these indexes is much more expensive than intersecting the result of a prior ai ^ aj operation together with ak. The main contribution of the paper is a new data structure, the view tree, that can be used to efficiently store and retrieve such prior results. These results, which can also be thought of as materialized views, can then be used to efficiently answer future queries. Note that object attributes could either be derived from application semantics (e.g. meta-data from files in a filesystem) or computed via techniques such as latent semantic indexing.